Hospital Readmissions by Variation in Engagement in the Health Care Hotspotting Trial

Key Points Question Were outcomes following the Health Care Hotspotting intervention different for those patients who were more vs less likely to engage with care management? Findings In this secondary analysis of randomized clinical trial data from 782 participants, greater intervention participation was associated with significantly lower readmission rates 30 and 90 days after hospital discharge and significantly lower 30- and 180-day readmission counts. Meaning These findings suggest that evaluation strategies that account for variable intervention participation can identify program effects that are missed in intent-to-treat analysis.

The hospital claims data that were used in the new analysis had been revised and updated since the primary analysis was published. 1Chi-square tests were used to test the independence between each outcome and the hospital claims data used in the two analyses.We found no significant difference across the treatment and control groups for each of the 30-, 90-, and 180-day readmission measures.

Primary analysis
New

Care Coordination database
Camden Coalition used its internal workflow database, built in TrackVia™, to record programmatic information including the enrollment and randomization steps, and all post-enrollment workflows, intervention encounters, and staff effort.Two data tables were extracted for this study: baseline survey data and client tracking data.Camden Coalition care team staff completed the baseline survey during the initial visit with the patient at the hospital bedside before randomization or enrollment occurred.The survey covered demographic, clinical, social, and health-related self-assessment information.A client tracking table, where all staff-patient interactions including the interaction type, date, and hours spent, was used to calculate intervention-level measurements of participation and engagement.

Arrest history
The Camden Coalition entered into a data sharing agreement with the Camden County Police Department in 2014.
The data shared by the police department with the Camden Coalition included name, date of birth, arrest date, location of arrest, and statute violation information for each arrest for the years 2014 through 2018.The arrest data were linked to the data described above.

eAppendix 2. Sensitivity analysis of engaged participation definition
Intervention participation is a key concept in understanding program implementation and is a central component of the distillation method. 2 Because the Camden Core Model had no a priori criteria for "engaged participation," we developed a definition for the purpose of this analysis, as described in the main paper.In summary, we considered three dimensions of intervention activity to develop a definition: 1. Number of intervention hours received during the first two weeks of intervention enrollment 2. Number of weeks with a successful intervention encounter during the first six weeks of enrollment 3. Length of time between enrollment and intervention outcome (i.e., graduated or lost-to-follow-up) The definition of engaged participation that we adopted is defined as anyone meeting at least two of the following criteria: 1. Received at least three intervention hours during the first two weeks of their enrollment 2. Had contact with staff at least once per week for four weeks out of the initial six weeks 3. Were retained in the program for 60 days (1/2 of average treatment length) or graduated within that timeframe To better understand how robust the study findings are to other thresholds for defining engaged participation, we performed sensitivity analysis by varying the above three thresholds.The algorithm used for sensitivity analysis was as follows: 1. Let the number of intervention hours during the first two weeks run from its 20 th percentile to 40 th percentile, namely, it takes values from {1.8, 2.3, 2.7, 3.3} 2. For each value in a, let the number of weeks with a successful intervention encounter during the first six weeks run from its 20 th percentile to 40 th percentile, namely, taking values from {3, 4} 3.For each chosen values from a and b, let the number of days to intervention outcome runs from its 20 th to 40 th percentile, namely, taking values from {53, 58, 66, 85} The 20 th and 40 th percentiles were chosen because many intervention patients actively participated in the program and graduated, but some patients received few intervention hours and were lost-to-follow-up soon after enrollment.These considerations suggested that the thresholds should not be set to the extremes of the distribution.The figures below plot the confidence intervals of odds ratios and incidence rate ratios for the 30-day and 180-day readmission rates and counts with varying parameter values of the engaged participation definition components.In each figure, the top 4 panels show the confidence intervals with a fixed number (3) of weeks with successful intervention encounters and a fixed number of days (53) to intervention outcome, but with varying intervention hours (1.8, 2.3, 2.7, 3.3) during the first two weeks.The number 5 and 6 panels in the middle show the confidence intervals with fixed number (1.8) of intervention hours and fixed number (53) of days to intervention outcome, but with a varying number of weeks (3, 4) with successful intervention encounters.Similarly, for the bottom 4 panels, the number of intervention hours and number of weeks with successful intervention encounters are fixed, the number of days to intervention outcome is varied.If the confidence interval is completely below or above the dotted horizontal line (y=1), then it is with solid circles to highlight significance.

Percentiles for each component of engaged participation
For both the logistic and Poisson model analyses, the results are consistent with those reported in the main paper.For all models, downward trends are observed with increased population distillation and the treatment effect becomes statistically significant when engaged participation concentrations of the population are in the range of 40% to 20% of total participants.These sensitivity analyses can be interpreted to show that our results are robust to changes in the definition of engaged participation.

Confidence intervals for incidence rate ratio of 180-day readmission counts at different levels of population distillation for varying values of each component of the engaged participation definition eAppendix 3. Model building Stage 1 Gradient boosting machine learning model
We built a gradient boosting machine model to estimate engaged participation among intervention patients.The variables included in the model are shown in the table below; all are drawn from the baseline period prior to enrollment.To find the optimal parameter values in the model, we used a line search algorithm defined as follows:

Variables included in the gradient boosting machine model
1. Learning rate runs from 0.001 to 0.01 with step increment 0.01 2. Number of observations allowed in one leaf runs from 5 to 10 3. Cross validation runs from 4-fold to 10-fold The performance parameter is the area under the receiver operating characteristics curve (AUC-ROC).The optimal combination of parameter values are the values for which the AUC is maximized in the test data obtained from the cross validations.Cross validation was used to prevent overfitting.The optimal parameter values are learning rate 0.001, 8 observations allowed in each leaf, 8-fold cross validations, which achieve an AUC of 0.82.The variables added to the model to estimate engaged participation included demographic variables, clinical variables (physical illness, behavioral diagnosis indicators, and past hospitalizations), and social variables.Two-way interaction terms among variables were also added in the model.

Stage Three Models Estimating Program Effects
We used logistic regression models to study 30-day, 90-day and 180-day readmission rates and Poisson regression models to study readmission counts.The formats of the independent variables and the inclusion of the variables in the final model were first determined based on bivariate analyses, then with a forward variable selection algorithm to select variables based on model performance metrics including the drop in AIC and/or p-values from Wald and likelihood ratio tests. 4,5riables included in the regression models RCT  a.Each disease on the Quan Index has a value ranging from 0 to 6, where 0 means no mortality risk and 6 corresponds to the highest mortality risk.The variable used in our models aggregates scores across the diseases on the Quan Index.We then dichotomized this variable into two groups: score lower than 3 and score greater than or equal to 3. b.The social scale is an aggregated score that measures a patients' social complexity.To create the scale, each of the following 5 attributes that were measured on the baseline survey was given a score of 1: less than a high school education, insufficient family support, unstably housed, unemployed, and unmarried (i.e., single, divorced, or widowed).

eAppendix 4. Poisson regression model validation
The Poisson regression model has a strong assumption of equidisperson that needs to be validated before model application.We calculated the average 30-, 90-, and 180-day readmission counts and then divided these by their corresponding variance with the raw data, plotted as a pink dotted line in the figure below.To correctly apply Poisson models, we limited the dispersion of our data by winsorization.If a patient's readmission count was beyond the 95th percentile of the data, we cut the value at the 95th percentile.The variance-to-mean ratio after winsorization is plotted as a blue dotted line in figure below and was improved with winsorization.

Poisson regression model validations: variance-to-mean ratio
To assess the integrity of the results after data winsorization, we reanalyzed the data using a robust sandwich estimator.The results from the two methods are similar and are shown in the table below.

eAppendix 5: Additional analysis
The primary results reported in the main paper are based on logistic and Poisson regression models.We also built Cox proportional hazard models to examine the time to first readmission.The data were right censored for patients with no hospital readmission within six months of their index hospital discharge.The first half of the table below displays the hazard ratio, the 95% confidence interval, and p-value based on the Cox models.The downward trend in the hazard ratio of intervention over control group becomes statistically significant at 20% population distillation (Hazard ratio: 0.64, 95% C.I. (0.42, 0.98), P=0.04).
Because Cox proportional hazard models only capture time to the first event, we built frailty models to capture the timing of repeated events.The second half of the table displays the hazard ratios, the 95% confidence intervals, and P-values at different levels of population distillation based on the frailty models.The declining hazard ratio becomes statistically significant at 20% distillation (Hazard ratio: 0.68, 95% C.I. (0.48, 0.97), P=0.03).
Both approaches yield results that are consistent with the main analyses presented in the main paper.

Hazard ratios of intervention over control group at different distillation levels X% of population selected
Hazard of hospitalizations in 6 months prior to trial enrollment, including the index admission during which the patient was enrolled in the trial.b.Flags whether the patient had an arrest by the Camden County Police Department in the 6 months prior to trial enrollment.

eFigure 2 :
Number of intervention hours received by treatment arm patients during their first week of intervention enrollmentThe data in the figure above cover 389 patients randomized to the intervention arm of the trial.Interactions that occurred between patients and care team staff either in person or on the phone were recorded in a care coordination database along with interaction length of time.The distribution ranges from 0 to 15 hours of engagement.Patients represented in the far-left bar received 1 hour or less of engagement during their first week of enrollment.

analysis Treatment group Control group Treatment group
a. Chi-square tests were performed to test the independence between each outcome and the hospital claims data used in the two analyses.

Engagement hours during intervention first two weeks No. weeks out of first six with successful engagement No. days to intervention outcome
Four patients indicated another race on the baseline survey (Asian, Multiracial, or Other).Because of modeling requirements, categories with only four elements could not be included in analysis.We therefore assigned these patients the most probable race and ethnicity category (Hispanic, Non-Hispanic Black, or Non-Hispanic White) such that the assigned category of each patient had the largest probability of association with their engaged participation outcome label.b.Flags whether the patient had an arrest by the Camden County Police Department in the 6 months prior to trial enrollment.c.Number of hospitalizations in 6 months prior to trial enrollment, including the index admission during which the patient was enrolled in the trial.
3.The medical diagnosis variables are diseases with a nonzero weight on the Quan index, which is an updated version of the Charlson comorbidity index.3

Intervention and control group patient characteristics within increasingly distilled samples
© 2023 Yang Q et al.JAMA Network Open.eTable 1